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SUMMARY 


Ao  Interest  grows  "Ni^lclly  In  Industry  on  the 
potentialities  of  mathematl^l  programming  techniques. 
It  appears  worthwhile  to  have  a^aper  devoted  to  some 
of  the  more  premising  c*evelopn;ents  vl\leh  may  speeci  up 
the  ti’ansltlon  from  Interest  to  use.  Tliree  topics 
have  been  selected  (in  three  sections  that  follow) 
v/t»lch  have  recently  cone  into  prominence : -w  urjcertainty, 

t 

combinatorial  problems,  and  large  scale  system:^.  ^  Hie 
reader  v/111  find  in  the  course  of  their  discussions  "• 
that  a  survey  —  though  perhaps  not  a  systematic 
survey  —  lias  been  made  of  current  techniques*  in 
the  linear  prograrruT.ing  field. 


•For  consolldatod  sources  on  techniques  in  linear 
programming,  see  flj  »  [^1 »  ,  and  fRlj  ;ror  in t:’Oductoi7 

material  see  [^]  and  [ijj , 


RECENT  ADVANCES  IN  LINEAR  PROGRAMMING 


by 

G.  B.  Dantzl;; 
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I.  UI.’CERTAINTY 

In  the  past  few  months  there  have  been  Important  developments 
that  point  to  the  application  of  linear  prograrnmln:;;  mothods 
under  uncertainty.  By  way  of  background  let  us  recall  that 
there  are  In  conu'.on  use  two  csEentlally  different  typer  of 
scheduling  applications— one  designed  for  the  short  rvyi  and 
,  those  for  the  long  run.  In  ruch  applications  the  effect  of 
probabilistic  or  chance  events  Is  reduced  to  a  mlnlnnr.:.  Hie 
usual  technique  for  doing  this  Is  to  pro'^lde  plenty  of  fat 
in  the  cystem.  For  example,  cor sumption  rates,  attrition  rates, 
v/ear— out  rates  are  all  planned  on  the  high  ride.  Times  to  ship, 
time  to  travel,  tines  to  produce  arc  always  made  well  above 
actual  needs.  Indeed,  the  entire  system  is  put  together  with 
plenty  of  slach  and  fat  v/lth  the  hope  that  they  will  be  the 
shock  absorbers  ’'/hich  will  pemlt  the  general  objectives  and 
timing  of  the  plan  to  be  executed  In  spite  of  unforeseen  events. 
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•Material  for  this  paper  has  been  drawn  from  speeches  delivered 
before  Hie  Institute  of  Minagement  Sciences,  Pittsburgh  meeting, 
October,  195^ »  and  the  Symposium  on  Linear  Programming,  Washington, 
D.C.,  January  25,  1955. 
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In  the  general  course  of  things,  long-range  planr  are  frequently 
revised  because  the  stochastics  elements  of  the  problem  have  a 
nasty  way  of  Intruding.  Por  thlr,  reason  also  tnr  chief  contri¬ 
bution,  If  any  of  the  long-range  plan,  is  to  effect  an  Irrunodjate 
decision — arch  as  the  appropriation  of  funds  or  th^  initiation 
of  an  Inportant  developr.ont  contract. 

For  ^hort-inin  scheduling,  r.any  of  th.e  elach  ani  fa^ 
techniques  of  Its  long-range  brother  are  employed,  principal 

differences  are  attention  to  detail  ar*d  the  short  1  inc— iioj-iron . 

Ae  long  as  vcapab  111  ties  are  v;ell  above  require  non  In  (oi*  dcj’^:inur ) 
or  If  the  derr.'indn  can  he  shifted  In  tlmc^  this  approach  presorts 
no  problem?;  l.e.,  it  Is  feasible  to  Inplornent  the  schedule 
In  detail.  Ho'*;over,  v;hero  there  are  .sljortages .  the.  jiro^cctcd 
plan  baaed  on  such  technlqucn  ruy  load  to  actions  far  from 
optlnvil ,  v/hereas  these  nev/  methods,  where  appllawSble,  'vxiy 
result  In  considerable  savli^gs.  T  shall  aubsta.ntlatc  this 
later  by  reference  to  a  problem  of  A.  .^orguson  on  the  r*cutlng 

f 

of  al  r<.  r..*.rt . 

Having  reviewed  the  need  for  technique?  that  more  effectively 
account  for  uncei’talnty,  let  us  turn  our  attention  nov:  to  a 
sequence  of  simple  Ideas  that  has  recently  culminated  in  this 
extension  of  linear  programmin';  methods.  To  Inltlito  the 
discussion,  a  special  application  of  the  "transportatlorj” 
problem  will  bo  described,  ["ib^  ,  flc]  ,  [2].  The  H/vND  Coi’poratlon , 
an  you  knov;,  has  extensive  computing  facilities  that  are  In 


^<5» 

«-lS^5 

» 

constant  use  by  the  research  personnel.  About  two  years  a^c 
the  author  was  consulted  with  regard  to  rindlnpr  an  Improved 
method  for  scheduling  work  requiring  computation.  The  need 
arose  because  the  computatlorj  iaoor*dtor:,'*o  "customers,”  the 
research  people,  were  unhappy  with  the  long  delays  gonerally 
incurred  between  the  time  a  request  for  computation  was  received 
and  the  their  vrorl:  was  completed.  Tl  o  fu'lticlpal  ooiiso 

of  dissatisfaction  was  quite  cleai*  for  there  wis  one  pi'Oject 
that  v/as  both  top  priority  and  so  large  In  volunr  that  Jt 
completely  absorbed  the  entire  computing  c;::pac3ty  for*  nary  v/eclcs. 

research  people,  bclnr  human,  were  no  jorgrT  Interes'fd 
Ir  the  computed  .answers  to  their  proMer.s  vhen  the  conputlnr 
lab  finally  got  around  to  them. 

In  this  example  we  hav'o  c  case  v;here  the  rolhod 

of  sched\illnfr  is  tiot  necessarily  the  best . 

In  order  to  develop  a  more  flexible  dec;’,  c*  or  r.x  tl  od  Maeri 
priority  schodullnr,  a  model  v/as  devlncd*  la  v/hlch  the  vn'aje 
of  a  Job  fell  off  the  longer  Its  completion  day  war.  jelayfjd. 

I 

Ihe  final  dete»’r.lns.tlon  of  the  optimum  .schedule — where  optlmam 

•'lYio  Model  of  Ootlmun  Scheduling  of  Projects  on  Punch  Car  ' 
Equipment  was  developed  by  Clifford  Shaw  of  RAUD  and  the  authox*, 
and  reported  Jointly  before  the  RAl'IIX-U. C. L. A.  Seminar  on 
Indu.^ trial  Scheduling  In  the  v/lnter  of  193^  (the  latter*, 

Incident 'illy,  being  one  of  the  foreninnei'C  of  The  Institute  of 
Management  Sclen'xc.h ) . 
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meant  ahat  echedule  whioli  tTave  ct.e  j^i  oate£.l.  '..oNii  'Joy''  to  Itr, 

customorc— depended  on  tlio  aolutlon  of  a  ti’ariaportatJon  prooaen 

-« 

v;ho8e  array  in  given  bolow: 


whe-o  ,  Ic  *1.<?  I'lorrr.  (to  bo  ioterr.I •".jiO  (v Jci 

—  V 

t  ^ 

In  the  j  *  wceb.  ?r  us  nonnoy.et !  vo  y  .  ^  to  ’.-c  '  r  \  )h  that 


«  I' 

=  (hours  rtsclr^nod  »  I  onro  rrqnJa'oc!  '  1''’  jol. ) 


T  ^  (f:oure  asol^iied  ^ 


V  biOtU'y  av''J  lal'le  in  .1  "  v;‘jni') 


and  such  that 
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I  I  *^1 

i  J 

where  c^j  was  the  value  to  the  customer  per  ^lour  expended  on 
hlB  project  In  the  j  week.  Topically  one  would  assign  less 
value  per  hour  the  longer  the  job  Is  delayed, •  l.e., 

=11  >  =12  >  •••  >  =ln* 

Now,  In  addition  to  these  restrictions  typical  of  a 
transportation  problem  this  application  had  the  added  wrinkle 
that 

°  ^  ’'ij  ^  ®ij 


V.Jl 

! 


•  -*1 


Which  states  that  the  hours  assigned  to  1^^‘  project  In  the  J^^ 
•week  cannot  exceed  Typically,  a^j  •  ^0  hours  meant  that 

only  one  person  could  be  assigned  to  job  J.  The  adding  of 
these  upper  bound  restraints  greatly  enlarges  the  size  of  the 
problem  if  one  proceeds  In  the  usual  manner  of  adding  extra 
equations  to  represent  these  restraints,  Ql]  ,  [^]  ,  [4— l]  ,  Q>]  . 


•TJ-ie  reader  will  recognize  that  putting  the  value  on  a  per 
hour  basis  rather  than  on  a  .lob  completed  basis  Is  a  dodige  v/hich 
an  experienced  fomulator  uses  to  get  around  Indivisibilities 
that  for  the  most  part  defy  mathematical  solution  (see  Section  II 
on  combinatorial  problems). 
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To  Illustrate,  a  problem  with  iS  projects  and  10  weeks  has  a 
total  of  28  equations  in  180  unknowns.  However,  after  the 
upper  bound  conditions  are  added  by  means  of  the  conditions 


'‘ij  ♦ 


ij 


(where  is  a  “slack”  variable,  [;i]  )  the  niuiioer  of  equations 
is  28  +  180  and  the  number  of  unknowns  is  2  x  loO.  Tiie  oonstaiits 
*ij  referred  to  as  "capacity  restraints  on  njutes." 

It  is  noc  difficult  (as  we  shall  show  below)  cnat  uhe 
enlarged  problem  is  still  a  transportation  problem,  alth.ough 
strangely  enough  x^j  now  appears  in  three  equatiOiir*.  If  t.’ue, 
the  values  of  in  an  optimal  solution  will  still  turn  out 
to  be  integers  if  the  h^,  and  are  integers.  To  chow  this, 
we  will  use  Orden's  transshipment  device,  [20] .  'IV;e  enlarged 
transpoi’tatlon  array  below  lllusti’ates  the  case  for  :a“2,  n*}, 

(the  procedure  Is,  oi'*  course,  general): 


f 


where  the  variables  In  the  array  sum  across  to  the  Indicated  row 

» 

totals  and  down  to  the  indicated  column  totals.  '  j 

' 

Ihls  constitutes  a  formal  proof  that  capacity  restraints 
on  routes  do  not  alter  the  character  of  the  problem.  It  Is  >} 

not  recorrmended  as  It  stands  as  a  short  cut  computational  device.  .v 

•  '  A* 

Our  purpose  now  Is  to  show,  however,  that  this  problem  can  be  >  ^ 

solved  with  only  slightly  more  effort  than  the  original  trans- 
portatlon  problem  without  upper  bounds.  The  procedure— referring 
back  to  our  original  array  Is  to  divide  the  variables  into  ''' 

three  classes: 

4 

(a)  "basic"  variables  (m*fn— 1  in  number)  whose  determinant  > 


is  nonvanishing; 


(b) 

(o) 


nonbaslc  variables  at  lower  bound  value  (0); 

nonbaslc  variables  at  upper  bound  value  (a^  .). 

*■  J 
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Tcr  cxar.iple.  In  tho  ari^ay,  wo  i.viy  indloalL-  Iasi? 

variables— by  pull  Inc  ^  cjnclo  aroiaiu  thetr.  ac  Is  01‘uen  doro, 
M»  [^]  i  TiOict,  place  an  ’'uppe.-'  bor  no"  f-rynbol  Tor  v:.r ‘.abl<.*o 
at  their  upper  bounds.  The  prices  ctnd  asuociat^^d  v;lt^ 
the  equations  are  computer  in  the  usual  manner,  j.e.,  -f  * 
for  all  basic  variables  The  criterion  for  optimality  for 

*  j 

such  a  system  can  be  shown  to  be: 


IJ 


u 


1 


^  ''j  ^  =ij 


for  all  r.onbaslc 
value; 

for  all  nonhaslc 
bound  value. 


variables 


varlabli;? 


at  j.owci*  bound 


^  at  upper 


If  any  variable  x^j  docs  not  satisfy  the  aoove  optlrr^llty 
critcrlo.n,  an  Improved  solution  may  be  obtained  hy  Increasing 
or  decreasing  It  until  some  variable  (perhaps  Itself)  cither 
hits  a  lower  or  upper  bourjd;  it  is  this  variucle  wi.icrt  Is 
dropped  out  frora  t’ne  basis. 

The  reason  why  this  device  v/orks— and  my  remar.es  now  apply 
to  the  moat  general  linear  programming  problem  and  not  just,  the 
transportation  type— lo  this:  The  simplex  method  divides  the 
variables  Into  two  classes  which  are  referred  to  as  b.?s3.o  and 
nonbaslc.  The  nonbaslc  variables  ai'e  customarily  sot  equal  to 
zero  and  a  i-ule  Is  given  as  to  when  It  pays  to  Increase  any  one 
of  the  nonbaslc  variables  from  zero  to  the  largest  value  possible 
which  preserves  feasibility.  A  little  reflection,  l.ov;eve:, 


v/1 1 1 


t  '•  s  V 
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make  it  clear'  that  the  iionbaoic  variables  could  l.ave  been  at 
any  constant  admissible  value  and  the  same  miles  \;ouid  c|:j  ly 
on  whether  It  pays  to  increase  their  value  atrJ,  cf  course,  the 
opposite  rule  applies  as  to  when  It  pays  to  decrease  It.  If 
the  Illative  cost  factcra  (e.g.,  —  v^)  are  other  tlrm 

zero,  then  depending  on  slf;f]  It  v;lll  alv/ayn  pay  to  I’lcrease  or 
decrease  the  quantity  of  a  variable.  However,  If  the  variable 
Is  at  either  end  of  its  ran.-^e  of  values  it  .may  not  t)e  i.osrlhlo 
to  do  Ih.is  without  losln/^  feasibility.  For  a  fullo!’  ncoouijt  of 
this  work  see  [j3J  and  the  CliHr’ies-Ltinke  paper  ! l]  . 

I^et  UR  now  lock  at  a  socorid  device  that  riiakos  effective  i:Ge 
of  this  upper  bouridlng  teciinlque.  Let  uo  consider  a  nitu^itlO'* 

It.  \/hlch  the  objective  function,  li.etead  of  beli.i:  a  ]li.Our  fonti 
to  be  minimized,  is  of  the  form 


(1) 


n 


7  P^(-xJ 


J-1 


'*'in 


(.%  >  0) 


wner’e  0  A\ .)  Is  a  convex  function  and  are  subject  to 

J  J 


{2) 


n 


J-1 


( 1*2  ,k,  .  .  .  ,rr )  . 


"Convex— separable"  is  the  term  used  by  Gharries  to  describe  this 
class  of  objective  fonris,  [‘*—11]  ,  also  [5]  .  trick  here  is 
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to  repreoent  aa  Indefinite  (not  definite)  Integral  v;hlch 

is  approximated  by  a  sum  over  k  fixed  Intervaln. 

To  Bee  this  clearly,  let  ur>  note  first  that  any  convex 
function  f^(x)  may  be  written 

X 

^(x)  -  / ^^•(u)du 

o 

where  fiix)  convex  means  p''(u)  la  nondecreaolng  at.d  that  p'('0 
may  be  approximated  by  a  histogram  over  some  Interval  0  <  u  <  c 
where  c  Is  some  assumed  very  large  upper  bovind  for  x.  In  or-der- 
to  avoid  any  discussion  about  how  v/cll  the  hlstograr:  fits  0'(u), 
which  Is  not  gerrtane  to  v/hat  follows,  we  are  actually  n.isuiTilng 
that  ^'(u)  has  been  I'eplaced  by  i  h.lotogram. 


Here  from  the  convexity  of  0.  rov;  I’c^iacc 

X  by 


X  «  +  Ag  + 


1 


■^.  '  /:4 
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where  are  noiinegatlve  varlatlec  and  k  Is  the  i'lxed  nuniber  of 
Intervals  In  which  the  range  of  x  wae  subdivided.  It  Is  easy 
to  see  that 


fi{^)  -  Min  J 
1 


0  <  <  a 


Indeed,  It  Is  clear  that  the  minimum  Is  attained  by  cl-.ooslng 

I  ^1  “  ^1*  ^  *  ^2*  ***  tmtll  the  value  of  x  Is  exceeded  for 
some  1  In  which  case  Is  set  equal  to  U.u  residual.  Tlius 
the  effect  of  the  minimization  Is  to  represent  0'(x)  by  the  area 
under  the  histogram  up  to  x. 

We  now  employ  this  approach  to  solve  (l)  and  (2);  the  pro¬ 
cedure  Is  to  substitute  for  variable  x,  and  In  the  linear 
^  J  J 

1  programming  problem  by 


k 

I 

1-1 


J— 1 , 2 , . . . , n 


and 


1-1 


and  noting  that  since  a  nlnlnum  for  T  cou^^ht  In-  (l), 

this  Implies  that  the  values  of  ,  satisfying  {j>)  must,  at  the 

X  ^  -  ■  T-r  •  1 

minimum,  satisfy  T  A,i  “ 
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Thls  manner  of  treating  convex  aepcirable 
appears  to  creat.ly  lnci*easc  the  number  of  varl...l  l^r.  ./-^bout 
inni'eaclrjg  the  number*  of  equatlo/js.  However,  tt  ia  the  nuiT.bcr 
of  equations  that,  as  a  rule,  determines  the  work  In  the  simplex 
method.  Moreover,  it  should  be  noted  that  there  are  numerous 
short  cuts  possible  due  to  the  appearance  of  several  columns 
with  identical  coefficients  (except  for  the  cost  row)  so  that, 
in  fact.  It  is  quite  simple  to  rapidly  solve  ca.sos  in*'olvlng 
a  convex  separable  objective  fom. 

Finally,  let  us  tuiT)  to  a  problem  Involvlr  ’•  uricc^rtalnty . 

For  this  puiTJOse  let  us  consider  by  way  of  an  exi*-.pie  the  c  ire 
of  a  cannery  that  has  several  factories  as  sources  arid  a  number- 
of  warehouses  as  outlets. •  The  typical  fonnulatlon  leads  to  a 
transportation  prx^blem. 


—  ^11  "  ^^1  “  avallabliity  at  1 oo-j.'Co) 

J-1 


Hi 


requirements  at 


rou 


e 


) 


m 

•I 

1«] 


n 

J=1 


Min 


cost  01'  nl  Ipplng  a  unit 
r rom  i  to  J )  . 


•For  '^xarnpio  H.  J.  flolnz  nianuf  a  a  tu  ro  s  ketchup  la  .a  iiair  J' 
plants  and  distributes  to  some  70  warehouses  scattered  throughout 
the  U.S.;  a  description  of  this  application  Herjuerson  and  S.hJ- 
wher't*  the  derands  wer^  assur.evl  knov/n,  is  given  in  [ijj  . 
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v’J-.  .. 

y  t 


th 


*  Suppose,  however,  that  the  requirements  at  the  J  “  source  are 
unknov.'n  but  are  given  by  some  sort  of  a  frequency  dlstrlbullor; 
Let  uf:  suppose  that  for  some  dectlnatlon  that 


J  X,  ,  -  u 


‘U 


is  the  amount  aaslgnod  and  v.  Is  the  actual  umount  der^nded, 

j 

which  of  course  occurs  latei’.  Then  the  revenues  v.-htch  tiie 


company  rc'^elvcs  v/111  be  proportional  to 


■ 


A 


Kln(u,,  vj. 
u  J 


It  Ir,  clear  that  the  ‘  expected  I'^venues  ire  a  function  of  thio 

amount  assigned,  u,.  Ln  fact,  if  .:q  let  0(u,)  je  the  expected 

J  J 

cor.tc  (i.e.,  I'cvcnucs,  if  negative)  tl.en 


^(u) 


■a  —  a 


r  u  00 

/  vf(v)dv  +  u  /  f(v)dv 

V/ 

v«0  v*u 


whore  a  Is  the  I'actor  of  rroportlonall ty  ard  r(v)cJv  Is  the 
j-robablllty  density  distribution  of  v.  It  Is  easy  to  see  by 
differentiating  that 


>  / 


0'(u}  - 


a; 

/  r(v)dv 


•f  , 
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and  hence  the  e::pected  cost  as  a  function  of  u  Is  convex. * 

Let  us  suppose  In  the  v;arehour.e  problem  that  all  demands 
at  destinations  are  subject  to  uncertainty,  and  that  the 
objective  Is  to  arranf?e  the  shipments  00  that  th.c  total  ex{)ectod 
costa  arc  minimized.  The  mathematical  problem  the),  becomes 

^  ^IJ  *  ^1  . . .,m) 

J-1 


I  -  Uj  (j-1,?, . . .,n) 

1-1 


where  now  the  expected  costa  are  given  by 


n 


I  2  °ij^ij  I 

1  J  J-1 


and  convex  functions  giving  the  exper  tod  rev  ernes 

at  J  If  Uj  is  assigned.  Prom  (b)  it  Is  clear  that  the  ciorivjtlve, 
may  be  approximated  by  a  hlstogr'iT.  obtained  from  tl.e 

j  J 

cumulative  distribution  of  l’(v)  starting  with  ^  a'>.  Accordingly, 
we  may  approximate  by  oui’  second  device  and  sec  th.it  ./o 

can  solve  this  problem  as  a  regular  ti-ansportatlon  problerr  with 


•Result  due  to  11.  Scarf,  TVie  HAND  Corporation,  see  [uj  . 
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upper  bounds  on  certain  variables.  Prom  a  practical  point  of 
view  the  work  Involved  in  introducing  uncertainty  into  a  problem 
of  tills  type  is  only  slightly  greater  than  before.  The  r'esultlng 
allocations  are  often  quite  different.  In  the  hypothetical 
airline  example  considered  by  Ferguson  and  me,  the  savings  in 
expected  costs  over  the  earlier  procedure  reported  In  L^l]  war 
about  lO,t'.* 

For  a  more  complete  account  of  the  application  of  linear 
programming  methods  to  a  class  of  problems  Involving  uncei’tainty 
the  reader  Is  referred  to  {G]  . 

II.  COMBINATOnIA L  PaOPfEMS 

Turning  our  attention  now  to  the  application  of  linear 
programming  to  combinatorial  type  problems  —  the  future  In 
this  area  appears  to  be  less  certain.  J^ecause  of  success 
with  the  "aosign.T.er, t"  problem*^  and  with  large  scale  "traveling 
salesman"**^  problems  it  docs  seem  to  be  worthwhile  to  try  to 


•Tlie  paper  on  the*  extensiot:  of  [llj  to  Inc  case  of  uncerValnty 

is  under  preparation. 

• 

••Ihe  Assignment  Problem  is  one  of  asolg.nlng  optlnially  m  me:, 
to  m  Jobs  v/hen  the  value  of  havlnm  tlie  1— th  man  on  th<'  J-th  Job  is 
Icnown.  It  is  a  special  case  of  a  transportation  problem  discussed 
earlier,  [lb],  [^IcJ  ,  [?]  ;  see  also  Votaw— Orden  paper  [?5j  , 

J.  von  i^euirann  t?2J  . 


***'Iiie  Tr^'voliiij;  .''jc It’srjLf)  Problem 
tc  lour  m  cities  so  as  to  ml.nimlzo 


Is  one  c!‘  I’lnJl.ng  tiio  best  order 
total  dlrtance  covered;  see  [9l • 


p-6se 
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flnd  further’  use  for  such  methods  In  this  area. 

It  i&  interesting  to  note  that  niathomatlcianf:  have  been 
looking  over  other  branches  of  mathematicn  as  well  .1th  the 
hope  that  they  may  find  al^^'nlf leant  appllcat  1  Oi.s  of  those  nev; 
n^thods.  Others  have  been  buoy,  not  only  enocura"’,ir.,;  applications 
In  different  branches  of  mathematics,  but  In  dllTeicnl  riclds 
such  as  chemistry,  economics,  engineering,  etc.  as  v;cll.  Perhaps 
one  I’eason  for  human  progress  arioes  fix>m  the  curiosity  of 
to  exploit  every  new  tool. 

An  example  that  has  recently  received  cone  attention  Is 
one  which  was  discussed  by  the  author  briefly  In  a  course  In 
the  Department  cf  Agriculture  Graduate  School  a  i  amber  oi’  years 
ago.  It  concerns  a  classical  problem  of  Ci»emlcal  TlicrTncdynamlcs : 
Given  any  mixture  of  gates  aider  constant  pressure  and  tc.mperatuie 
conditions.  It  will  eventually  reach  an  equilibrium  position; 
problem  •—  detemnine  the  amounts  of  various  types  oi‘  molecules 
In  the  mixture  'when  the  equilibrium  Is  r’oached.  'Hits  type  of 
problem  can  be  I'eprenented  in  mathematical  tenr.s  In  the  Toimi  of 
a  system  of  equations. 


V  (Xj  >  0) 


^  Xj  -  X 

4 

C/ 

2^  X,  log  (YjXj/x)  -  Min 
J 


V.’*' 


number 


where  Xj  =*  number  of  moleculec  of  the  J  type; 

t>  w 

of  atoms  of  type  1  in  a  molecule  of  the  type;  b,  -  niinber 

of  atoms  of  type  1;  y ,  Is  a  given  constant.  Tlic  form  being 

j 

minimized  Is  called  the  free  energy  function.  Qienlstr  solve 
such  syr  terns  In  a  fairly  efficient  v;ay.  They  start  out  with 
that  combination  of  nv-type  molecules  which  form  the  major  mole 
fractions  in  the  mixture.  The  first  approximate  solution  can 
be  obtained  by  setting,  as  first  estimate,  the  minor  components 
zero;  from  this  first  approximation  of  major  components  an 
Improved  estimate  on  minor  components  can  be  made  and  the  process 
can  then  be  repeated.  Ct  jmlsts  are  lnter*eoted,  however,  In 
being  able  to  solve  such  problems  without  the  assumed  prior 
knowledge  of  v/hlch  combination  constitutes  the  major  components. 

Recently,  Selmer  Johnson, •  observing  that  the  free  energy 
function  is  a  convex  function  In  the  nuiriber  of  molecules  of 
different  types  In  the  mixture,  i/as  able  from  this  to  set  up 
a  procedure  that  extended  many  Ideas  fou.hd  In  the  separable 
convex  case  discussed  earlier  to  the  case  of  a  more  general 
convex  function.  His  procedure,  of  course.  Is  free  of  any 
assumption  about  major  components. 

Now,  my  objective  In  discussing  this  problem  In  this  section 
Is  simply  to  point  out  that  there  is  a  combinatorial  aspect 
contained  in  every  linear  programming  problem.  Indeed,  the 
basic  problem  Is  one  of  selecting  from  tie  class  of  extreme 


•RAND  Corporation  paper  In  preparation. 
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points  of  a  polyhedral  convtx  the  one  which  maximizes  a  given 
linear  form.  The  fact  that  there  are  procedures  like  the 
simplex  method  which  are  fairly  efficient  In  selecting  such 
combinations  Is  the  reason  why  It  ic  tried  for  certain  com¬ 
binatorial  problems.  Indeed,  it  Is  just  those  problems  where 
the  extreme  points  of  a  convex  can  be  identified  with  the 
combinations  of  Interest  v/here  this  approach  has  paid  off.  In 
the  case  of  the  traveling  salesman  problem  it  was  necessary 
to  go  further  and  to  find  ways  of  removing  extreme  points  of 
a  convex  which  could  not  be  Identified  with  tours.  In  the  case 
of  the  aaslgnment  problem  this  was  not  necessary. 

Let  us  turn  to  topology.  Recently,  Alan  Hoffman*  was  able 
to  u.se  linear  pr»ogranf:dng  to  prxjve  an  Interesting  theorem  of 
Dllv/orth  on  partially  ordered  sets— namely ,  that  the  maximum 
nu.mbe!'  of  elcnentc  with  the  property  that  any  pair  of  elements 
In  the  .cubset  ai'e  unrelated  is  equal  to  the  minimum  ri'amler  of 
disjunct  chains  covering  the  set.  What  he  showed  v.-as  that  this 
thoorem  was  our  old  friend  the  Cviailty  theoren  r,C  linear  Ir*— 
equality  theory  In  disguise.  Sr^irnulatcd  by  this,  Loth  'ioffman 
and  Independently  Fuli'erson  (joint  wit*  the  auth.or),  [Cl,  were 
able  to  .show  that  a  theorerr  due  to  Ful. '.arson  ano  Ford  o.u 
cap'j'^ltloG  1n  netwrrh"  ..'as  a/al,''  our  eltj  friend  the  duality 
theorem  In  disguise.  Tlicre  Is  a  close  .’^elation  between  this 
theorem  and  i  vvel  1— iLno'T)  theoi'err.  of  Menger  on  graj^hs.  TliO 


h’.’ational  Bureau  of  Standards  paper  In  preparation. 
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Pulkercon— Ford,  Max  Flow  Mlri  Cut  T^ieorefn,  [12],  states  that  the 
n;ixtr&al  flow  between  a  fixed  source  and  a  fixed  sink  in  a  network 
is  equal  to  ir.lnimuin  cum  of  capacities  on  arcs  which  separate  or 
cut  the  source  fi'om  the  clnk. 


Almost  all  ccr.blnatorlal  applications  that  I  fomlllar 
with  In  this  area  dcpond  on  the  Integral ’character  of  the  bade 
solutions  of  the  trajisportatlon  pi'ohlorr..  Unleoc  other  types 
of  ’no trices  can  be  discovered  with  thlc  property  for  their 
basic  solutions  or  at  least  for  the  optimal  basic  oolutlons 
(as  In  Karkov/ltz'  Metal  Procccs  Model,  )  the  potential 
developncnts  In  the  conblnatorlal  field  ’nay  b^  llrrltecl.  I  have 
found  one  case  whei’e  the  optimal  values  of  variables  wei'e  Integers 
In  a  linear  progranmlng  problem  wtilch  v/ao  not  of  the  transportation 
type  In  a  puzzle  vrt^.lch  I  recently  r'ln  across .  Jacobs  I'eportc 
that  his  "Caterer  Model"*  has  this  property.  This  gives  hope 
that  the  field  ;nay  net  scon  be  sterile. 


it- 


•The  Caterer  Problem  concerns  the  scheduling  of  purchases  and 
lauriuerintj  of  fresh*  napkins  for  a  known  future  schedule  of  meals; 
It  18  a  paraphaned  aircraft  spare  engine  problem,  [15I  . 
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III.  SPECIAL  METHODS  FOR  3QLVINQ  LARGS  SCALE  SYSTEMS 

last  lectlon  Is  devoted  to  a  short  plea  that  linear 
programmers  pay  greater  attention  to  special  methods  for  solving 
the  larger  matrices  that  are  encountered  In  practice.  With 
regard  to  the  pooolblllties  of  solving  large  scale  linear  pro- 

M 

graiopilng  problems,  c ^e  can  sound  both  an  optimistic  and  a 

> 

'  pessimistic  note.^  Ihe  pessimistic  note  concerns  the  ability 
of  the  problem  foimulator,  either  amateur  or  professional,  to 
develop  models  that  arc  large  scale.  The  pessimistic  note 
also  concerns  the  inability  of  the  problem  solver  to  compute 
models  by  general  techniques  when  they  are  large  scale.  If 
this  Is  so.  Is  not  thf  great  promise  that  tiie  linear  programming 

r 

approach  will  solve  scheduling  and  long  range  planning  problems 

with  eubstantlal  savings  to  the  organizations  adopting  these 

) 

methods  but  an  Illusion  and  a  snare?  Are  the  big  problems 
going  to  be  solved  as  they  have  always  been  8o}vod-~by  a  detailed 
system  of  on— the— spot  somewhat  natural  set  of  priorities  that 
resolve  every  possible  alternative  as  It  arises? 

Lot  us  consider  a  modest  planner  v/ho  Is  concerned  with 
the  expansion  of  motor  productlorv— let  us  say  a  special  type 
motor  that  requires  a  special  type  of  steel  and  must  use  tools 
fabricated  from  this  steel  and  the  tools  which  fabricate  these 
tools  also  use  this  steel.  Ihe  tools  that  fabricate  steel  we 
will  call  below  steel  capacity,  those  that  fabricate  tools  — 


t 


tool  caF>aclty,  and  those  that  fabricate  motors  —  motor  capacity. 
The  planner  is  quite  modest  because  he  Is  willing  to  consolidate 
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all  of  the  multitudinous  activities  and  items  Into  these  simple  v 

ft.’- 

terms.  The  Initial  Inventory  must  satisfy  the  first  5  equations  i 

In  detached  coefficient  form  given  in  the  tableau,  while  the  outputs 

from  the  activities  In  first  time  period  must  satisfy  the  next  5 
equations.  ..  ' 


Actlvitlep  (Ist  Period)  Actlvltiea  (Pnd  Period) 
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If  the  planner  la  Interested  in  developing  a  program  over 
two  years  by  quarters  that  meets  a  specified  schedule  of  known 
sales  and  creates  the  largest  stockpile  of  motors  for  any 
future  sales  that  may  develop,  then  the  pattem  of  coefficients 
In  the  tableau  must  be  repeated  for  eight  time  periods.  If 
we  denote  the  upper  and  lower  blocks  by  A  and  B  respectively, 
the  model  has  the  fozm 


(1) 


5x8 


10  X  8 

A 

B  A 
B  A 

B  A 
B  A 
B  A 

P  A 
B  A 


TYie  resulting  system  of  jj^O  equations  In  ^  variables  with  the 
objective  to  maximize  a  stockpile  of  motors  be  solved  In 
a  half  hour  on  a  modem  electronic  computei’.  Let  this  planner 
now  decide  that  his  model  is  entirely  too  coarse  and  chat  he 
must  plan  by  months ,  distinguish  two  types  of  iiiotors  and  two 
types  of  steel  and  our  resultant  system  becon*es  7  x  r;4,  14  x  24 
or  164  X  ^^6.  At  this  size  the  computation  would  require  about 


one  week  using  one  clilft  per  day.  Pporn  the  viewpoint  or  the 
computer  the  planner  is  no  longer  modest,  However,  for  the 
planner  It  is  clear  that  the  so-called  "detailed”  model  above 
is  at  best  only  useful  as  an  ovei'-all  type  of  guide,  but  hardly 
detailed  in  a  realistic  sense. 

Let  me  cite  an  exaniplt  from  another  area  —  the  problem  of 
routing  cargo  aircraft.  Let  the  variable  represent  the 

number  of  aircraft  of  type  k  routed  between  city  1  and  J.  Let 
us  distinguish  between  six  types  of  alrci*aft,  ten  time  periods, 
and  twenty  cities.  In  addition,  consider  a  second  set  of  variables 
^IJi  tons  of  cargo  shipped  between  city  1  and  J 

on  the  way  to  i.  Our  equations  become 


Aircraft  in  -  Aircraft  out:  J  ^cjk  *  ^  ^Ick 

J  1 


(k-1, . . .,6)(c-l, . . . ,20) *  ' 

'  i 


j 

Cargo  In  -  Cargo  out:  )k“  ^ci  (k-l.  ■  •  •  .20)(o-l, . . .  ,20)  ^ 

J  1  1 

Itamg*  Cap.  2  Tonnag*  Req.;  J  ’  I  tf  . 20)0-1 ,20) 

k  i  .*5 


Plane  Months  Avnllable:  7  7  »•  •  Pi, 

1  J 


As  we  see  again  ouch  a  system  involving  only  a  few  cities,  type 
aircraft,  and  cargo  destinations  generate  easily  a  system  in 


1,000  equations  lii  10, OCX)  uiiknowns.  Superflcliilly ,  a  vei-y 
discoux^oglng  situation. 

Indeed,  at  the  present  time.  It  lo  possible  to  solve 
rapidly  problams  in  the  order  of  a  liundred  equations.  Tho 
Orchard— Hays  701  Simplex  Code,  flfl  »  has  solved  raany  problems 
of  this  size  with  as  high  as  1,500  unknowns  and  machine  times 
of  five  to  eight  hours  as  a  rule — all  with  excellent  standards 
of  accuracy.  However,  It  lo  self-evident  that  no  matter  how 
much  the  general  purpose  codes  are  perfected,  they  vlil  be 
unable  to  solve  the  next  generation  of  probiens  which  will  be 
larger  In  size. 

However,  let  us  note  there  has  been  progress  In  this  area: 
The  excellent  vrork  of  Jacobs  on  the  caterer  proLlori,  [15]  ,  and 
the  work  of  Jacobs,  Hoffman,  >  «^':hnaon,  fhe  prx>— 

ductlon  smootl.lng  problem  ai'e  examples  of  what  may  be  acne  with 
certain  dynamic  models  wltli  a  slraple  I'cpetlt.lve  stiuctui't?. 

Cooper  and  Charnes  Jiave  employed  In  tb«lr  work  a  nuui-Ler-  of  short 
cuts  that  have  permitted  resolution  of  oc-rtaln  scale 

systems.  At  RAIJD  we  have  found  efficient  v.-ays  to  Viand  compute 
generalized  transportation  problems,  arid  MirkovM  ♦  r  has  proposed 
Q  general  pr%)cedure  In  this  area  that  is  pi  or.l r.lrig .  Many  mocieln 
exhibit  a  block  triangular  structure;  and  certain  partitioning 
m.othois  have  been  proposed  which  take  acivaritaiJc  of  tnis  type  of 
structure,  [5]. 
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At  RAND  there  Is  a  special  electronic  computer  code  for  a 

special  type  of  problem  called  the  "metal  processing"  model, 

Cl8] .  Mathematically,  this  may  be  described  as  a  slightly 

# 

generalised  type  of  transportation  model  of  the  form 
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I  *1J  ”  ‘’j 

J— 1 ,2,...,n 
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where 

^iy  ^1*  ^IJ  known 

constants  and  x^j  >  0. 
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H^e  special  code  (which  only  works  for  a  particular  choice 
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)  can  work  out  solutions  to 

a  300-equation  system  of  this 
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type  In  about  an  hour. 

Except  In  highly  specialised  models  of  the  transportation 
type  or  others  where  unusual  characteristics  can  be  taken 
advantage  of,  It  Ic  suggested  for  Industrial  applications  that 
every  effort  be  made  In  the  early  stages  to  prepare  a  consolidated 
version  of  a  model.  There  are  several  reasons  for  this.  In 
the  first  place  this  effort  results  In  a  model  which  Is  often 
very  useful  In  Itself.  Secondly,  It  provides  an  excellent 
dry  run  for  methodology.  In  a  word,  it  Is  better  for  adminis¬ 
trative  and  technical  I'eaoons  to  keep  the  model  Initially  small. 
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Inevitably  when  the  size  of  models  does  Increase,  there 
are  a  number  of  devices  that  give  promise  of  greatly  reduclrc 
the  amount  of  computation.  Het/ever,  conoldorably  r.oro  n=3oar  ' 
than  has  occurred  to  date  Is  needed.  In  the  first  place  thor<' 
appears  to  be  a  number  of  Important  characteristics  commonly 
found  In  practical  models  that  need  to  be  exploited: 

(1)  Most  factors  in  the  coefficient  matrix  are  sere, 

(2)  In  dynamic  structures  the  coefficients  ai-e  often  thi 
same  from  one  time  period  to  the  next. 

(3)  In  dynamic  solutions  the  actlvltleo  employed  often 
persist  from  one  period  to  the  next. 

(4)  Transportation  type  submatrlces  arc  common. 

(5)  Block  triangular  submatrices  are  common. 

Block  triangularity  Is  one  of  the  most  promising  character^ 
Istlcs  to  exploit,  [5]*  When  a  matrix  Is  composed  largely  of 
zeros  where  the  zeros  are  in  no  obvious  pattern,  it  Is  often 
practical  to  solve  directly  for  the  prices  and  the  representations 
of  the  vectors  entering  the  basis  rather  thaji  to  solve  for  them 
by  means  of  the  Inverses  of  successive  bases.  The  transportation 
model  Is  a  classical  case  where  this  approach  has  paid  off. 

When  many  variables  have  simple  upper  bounds.  It  Is  no  longer 
necessary,  as  we  have  seen  earlier,  tc  add  one  more  variable 
and  equation  for  each  such  restraint.  Instead,  it  Is  possible 
to  slightly  modify  the  original  simplex  algorithm  and  apply  It 
to  the  system  excluding  the  upper  bounds. 


In  many  problems  there  are  equations  that  ntay  be  considered 
as  foxing  a  set  of  side  conditions;  for  example,  conditions 
that  capacity  of  certain  machines  Is  never  exceeded  or  the 
characteristics  of  a  certain  product  (e.g.,  viscosity  Is  within 
specifications).  In  most  problems  only  a  small  subset  of  these 
"secondary"  constraints  are  likely  to  be  active,  l.e.,  at  their 
critical  value— the  others  being  well  within  capacities  or 
specifications.  In  such  cases  It  Is  reconrnended  that  the  linear 
programming  problem  be  first  solved  without  regard  to  these 
secondary  constraints;  then  the  system  Is  enlarged  to  include 
the  secondary  constraints  and  an  initial  basis  is  obtained  by 
augmenting  the  final  basis  of  the  smaller  problem.  This  will 
result  In  a  basis  in  which  not  all  variables  associated  with 
the  secondary  constraints  are  positive.  However,  l.n  this  form 
the  dual  simplex  procedure  of  Lemke  may  be  employed,  [17]  . 

Often  In  practical  cases  only  a  few  Iterations  ai^  needed  to 
clean  up  the  negative  variables  and  obtain  an  optimum  solution. 
Pot'  IXilier  discussion  of  these  possibilities  see  [^J  ,  f7|  . 
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